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Fitting y=fix When the Variance Depends on x 
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This paper presents some results concerning the selection of a method for estimating the slope 
of a straight line through the origin. For fitting the line y = ftx when the variance of y is proportional 
to x p , it is well known that the best estimate of ft depends on p. In practice, however, only integer 
values of p would be convenient to work with. One of the estimators appropriate for p = 0,1,2 would 
probably be used if the value of p were in fact fractional, or if it were only approximately known. This 
paper provides some guides for choosing the best among these estimators in a particular situation. 

Formulas for the best estimators of ft and their variances are given. Estimators of ft appropriate 
for integer values of p are compared in the case when p is not integral, but is known, and in the case 
when p is only approximately known. Estimation of the variances of estimators of ft is considered. 
Finally, some results are given on the effect of the spacing of the x values on the comparison of the 
estimators. 



This paper presents some results concerning the 
selection of a method for estimating the slope of a 
straight line through the origin. For fitting the line 
y = (3x when the variance of y is proportional to x p , 
it is well known that the best estimator of /3 depends 
on p. In practice, however, only integer values of p 
would be convenient to work with. One of the esti- 
mators appropriate for p = 0,1,2 would probably be 
used if the value of p were in fact fractional, or if it 
were only approximately known. This paper provides 
some guides for choosing the best one in a particular 
situation. 

The line y = /3x is to be fitted to observed points 
(xu yi), • • • , (x n > y«), where %\ > 0. The values of x 
are assumed to be known without error. For fixed 
Xu the corresponding y,, is an observed value of the 
random variable F, with mean fixi and variance 
V(Y i ) = a 2 x i i>,p^0. 

Let B p denote the "best," i.e., the minimum vari- 
ance unbiased linear, estimator of (3 when V(Yi) = cr 2 Xi p , 
and let V P (B P ) be its variance. Denote by V p (B r ) the 
variance of the estimator B, when V(Yi) = v 2 Xi p and 
p 7^ r. When r is restricted to the integers it will be 
represented by k. 

Section 1 contains formulas for estimators of j8 and 
for the variances of these estimators. Comparisons 
of Bk and Z?a+i are considered in sections 2 and 3, with 
numerical calculations given for the case of equally 
spaced values of x. Section 2 is a comparison of Bk 
and Bk + i (A integral) when p is known and k < p 
<k'+l. Values oip*=p n {k, k + \) are given such that 
when k^p<p*, Bk is better than Bk+u an d when 
p* < p ^ fc + 1, then Bk+i is better. In section 3, Bk 
and Bk+i are compared when it is known that k < p 
< jfc+l, but the value of p is unknown. Comparison 
of the relative efficiencies of these estimators shows 
that B k is better than B k + \ for k < p < fc-f 1. In sec- 



tion 4, an estimator, t r , of the variance of B r is con- 
sidered. It is an unbiased estimator oiV p (B r ) whenever 
r = p and for all p when r=2. It is positively biased 
when p < r < 2, and negatively biased when r < p < 2. 
Section 5 contains some results on the effect of the 
spacing of the x values on the comparison of estimators. 
Methods for fitting a straight line when the variance 
depends on x are treated in a general way in, for 
example, Brownlee's chapter on linear regression. 2 
The present paper provides further details relating 
to the choice of weights, for a particular case of the 
problem considered in Brownlee's section 11.14, 
"Weighted Regression through the Origin." The 
methods employed here could be adapted for con- 
sideration of other cases. 



1. Formulas for B P and V P {B P ) 

Let Y\, Y 2 , . . . , Y n be mutually independent ran- 
dom variables corresponding to a set of exact positive 
values Xi, #2, . . . , x n , with expectations 

E(Yi)=/3xu 

and let the variances V(Yi) be proportional to x/*\ 
i.e., let 



V(Yi) = <r 2 xf, 



0. 



Then the best estimator B p of /3 is the value of B p 
that minimizes the sum of weighted squared residuals, 

M (T 2 xf 
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The estimator is 



B P = - 






(1-1) 



Its variance is given by 



Vp(B P )=- 



(1-2) 



2 X *- P 



Table 1 gives formulas for B p and V P (B P ) for p = 0, 
1 and 2, when there are n points: (#i, yi), (#2, y2), • • ■ , 
(*n, yn). 

Table 1.— Formulas for B p and V P (B P ) 
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If there are m values of y for each x, then y* in the 
above formulas is replaced by yij with j= 1, 2, . . . , #i, 
and the denominator of each estimate and variance 
has the additional factor m; e.g., for p = 1, 

m n 

2 2 w 

»1 — n 



Yxi 



and 



V 1 (B 1 )=- 



<T< 



% Xl 



It has been assumed for convenience that all x% are 
positive. In general, one might have both positive 
and negative values of x. The above formulas are 
easily modified for use when the variance depends on 
the absolute value of x, 

V(Yi) = a 2 \xi\p. 

Alternatively, the formulas in table 1 may be used 
directly provided every pair (xu yd having x% < has 
been replaced by (— xu - yO- 



2. Comparison of B k and B k+X (k Integral) 
When A<p</c+l 

Since numerical calculation of B p and its variance 
is not easy when p is not an integer, the question arises 
as to which estimator, Bk or B k +u is the "better" esti- 
mator when p is known but lies between the integers 
k and k + 1. Regardless of the true value of p, B k is 
an unbiased estimator of f3. Thus the question of 
which of the two is the "better" reduces to the ques- 
tion of which one has the smaller variance. This can 
be answered if we can find a value p* such that 
/c<p*<A:+l and 

V p (B k )<V p (B k+1 ) 9 k^p<p* 

V p (B k ) > V p (B k+1 ) , p* <p ^ k+ 1. 

Here V p (B k ) denotes the variance of Bk calculated 
under the assumption that V(Yi) = cr 2 Xi p . In general, 



o- 2 Y Xi 



2-2r+p 



V p (B r )=- 



n "12 

L ;=i J 



(2-1) 



The value of p* may be found by setting V p {B k ) equal 
to V p {B k+1 ), and then solving for p. 

Values of p* have been found for the case of equally 
spaced positive values of x; that is, xt = ic, £ = 1, 

2, . . . , n, and c is any positive constant. The line 
y — ^x is then fitted to the points (c, yi), (2c, y>), . . . , 
(nc, y n ). Let p n (k, h + l) denote the solution of the 
equation V p (B k ) = V p (B k +i) in the interval k<p<k+l. 
Values of p n {k, k -hi) were found for k — 0, 1, and for 
various values of n. For small n (2 ^ n ^ 10), p n {k, 
k+l) was found by graphical methods. The exact 
values of p n { k, &+1) for k = 0, n = 2 and k = l, n — 2, 

3, were also obtained directly by solving the equation 
V p (Bk) — V p {Bk+\). These results agree well enough 
with the graphical results so that it is believed that 
the results obtained graphically are correct to within 
one unit in the second decimal place. The value for 
n=100 was obtained by approximate methods. For 
n — >co ? p^k, k-\-\) was found by letting n tend to 
infinity in the equation V p (B k ) = V p (B k + 1) and then 
solving the resulting equation for p. The values of 
p n (k, k + 1) are given in table 2. Since the range of 
p n (k, k + 1) is comparatively small over all values of 
n, we state a convenient rule: 



When 


Use 


0<p<0.6 


fio 


0.6 <p< 1.6 


Bi 


1.6<p<2 + ? 


B 2 



Table 3 contains formulas for the estimators Bk (k 
= 0, 1, 2) for equally spaced x and gives their variances 
V p (B k )foip = 0, 1,2. 

Figure 1 and figure 2 give, for n = 3 and n=10 
respectively, values of V P {B P ) and V p (Bk) for p rang- 
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Table 2. Values of p n (k, k + I) for equally spaced x 
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k = 


k=l 
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Table 3. 


Variances V p (B k ) and slopes B k for equally spaced x 
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ing from 0.0 to at most 3.0 and & = 0, 1, and 2. Figures 
such as these (on a larger scale) were used to obtain 
the values of p n (k, k+l) 9 2 ^ n ^ 10. For convenience, 
the values of c and cr 2 were chosen equal to one. It 
is evident from the two figures, and it also can be shown 
mathematically, that V P (B P ) and V p (Bk) are increas- 
ing functions of p when all values of x are at least 
equal to one. 



3. Relative Efficiency of B k/ B k+l 
When A<p<A+l 

Suppose it is only known that the value of p is in 
the interval (&, k + 1). Which estimator of /3, Bu, 
or fl/b+i, is then the "better" of the two? One way 
to answer the question is to use the idea of relative 
efficiency. The efficiency of an estimator T\ of 6 
relative to an estimator T 2 of is defined to be 



R.E.(r,|r 2 )=100 



E(T 2 -6) 2 
E(T X - Of 



expressed as a percentage. If R.E. [T x \ 7V) > 100 per- 
cent, T\ may be considered the better estimator of 
6. If E{Ti) = E(T2) = 6, the relative efficiency is the 
ratio of the variances, 



R.E.(Fi|r 2 ) = 100 



V{T X ) 




Vp(Bp) 



Vn(Bn) 



V p (B,) 



V p (B ) 



J I I L 



Figure 1. Variances V p (B k ), V p (B p )/or c = 1, & 1 = 7, n = 3. 
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P 

FIGURE 2. Variances V p (B k ), V P (B P ) for c = 1, a 2 = 1, n = 10. 
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To compare B k with B k + u we consider the least 
advantageous situation for each: (1) let B k be the 
estimator when in fact p = k+l, and (2) let B k + 1 be 
the estimator when p = k. Under these conditions 
the best estimator can be found by comparing two 
particular relative efficiencies: 

(1) the efficiency of Bk relative to Bk + i, assuming 
p = £ + l, 

R.E.(fl^ + 0= F * + i y , (3-1) 

relative to B k , assuming 

V k (B k ) 



(2) the efficiency of B k + i 
p=k, 



R.E.(B k+1 \B k ) = 



V k {B k 



(3-2) 




Figure 4. Relative efficiency curves for n = 10. 

R . E . W , =100 ^|>] = £ 



Both of these efficiencies are less than 100 percent, 
because the numerator is the variance of the esti- 
mator whose variance is minimum. However, if 
one of these relative efficiencies is larger than the 
other, then the estimator whose variance is the 
denominator of this larger ratio may be considered 
the better of the two estimators in the sense that it 
is less inefficient under the worst possible circum- 
stances. A comparison between such pairs of rela- 
tive efficiencies was made for the case of equally 
spaced x when k = and k=l. This comparison R- E - (Bi\B ) is less than the minimum of R.E. (S |#i 
of the relative efficiencies showed that for both values m tne interval =^p ^ 1. 
of A:, 

R.E. (B k \B k+1 ) > R.E. (B k+i \B k ) 4 - Estimation of the Variance of B r 

whenever n ^ 2. Thus, B is better than Si when ^ 
^ p ^ 1 and B x is better than B 2 when 1 ^ p ^ 2. 
Figure 3 and figure 4 show for n = 3 and n = 10 
respectively values of 



It has been shown that the variance of estimator 
• is 

.2 



Vr(B r )=- 



cr" 



R.E.(B,|B fc ) = 100 



VJBjc 



2> : 



2-r 



IVtiBdy 



for all p in the interval 0.0 to 3.0 and k and / equal 
to some combinations of 0, 1, and 2. To verify the 
conclusion stated in the previous paragraph, notice 
for example that (in both figures) Fo/Fi at p = is 
smaller than Vi/Vq at p = l; that is, the minimum of 



Let t r denote the following estimator for the variance 
of£ r : 

tr == ~Z * 



(4-1) 
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where S£ is an estimator of cr 2 , 

o 2= 1 ^ {yi-BrXif 



o, 



or, in a form more convenient for computation 



Sl = - 



1 



h J n 



*r 



2.0 



Figure 3. Relative efficiency curves for n = 3. 

\V P {B k )-] _ V k 
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When V(Yi) = a 2 Xi p , the expectation of t r is 

j^~ r JT^ 2 " 2 ^ 

E p (t r ) — " 



R.E. (B,|5 fc )=100 



.^(fii) J F/ 



Y*»-* 2> 2 - r 



(4-2) 
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When p = r, t r is an unbiased estimator of V r (B r ). 
When p # r, one would like the expectation,/^^), to be 
Vp(B r ) as given in eq*(2-l), but this is not usually the 
case. However, when r = 2, t r is an unbiased esti- 
mator; i.e., Ep(t2)=Vp(B 2 ) for all p. The bias of 
estimator t r is the difference, Ep(t r ) — Vp{B r ), and this 
quantity is positive or negative depending upon the 
values of p and r. In general, for positive values of x 
which are not all equal, the bias of estimator t r is posi- 
tive if p < r < 2, and it is negative if r < p < 2; i.e., 



E p (t r )>V p (Br\p<r<2, 

Ep(tr)<Vp(B r ),r<p<2. 
A sketch of the proof of this result follows. 

/ n n n 

£ Xi p- r y %i 2 - r -ny xt 2 - 2r+ p 



E p (t r )-V p (Br) = 



n-\ 



$xt 



(4-3) 



which can also be written as 



cr^ 



1 



n — l^H 



Consider the product (x, p - r — xf~ r ) (jCj 2_r — x/ 2_r ); and 



take xi ^ JC2 



\ x n , but not all equal. 



(1) If p < r < 2, the signs of the two factors are 
(+) (+) for all pairs Xu Xj for which #* 7^ #j. The sum of 
such terms is positive and hence the bias is positive. 

(2) If r < p < 2, the signs are (— ) (+) for all pairs 
Xu Xj where x\ # Xj, and in this case the bias is negative. 

Thus the estimator t r is likely to overestimate the 
variance V p (B r ) if r is such that p < r < 2, and it is 
likely to underestimate Vp{B r ) if r < p < 2. 

For the case of equally spaced x, the effect of 
differences between p and r is shown in figure 5, where 
the relative bias of estimator t r , 



R.B. = 100 



E p (t r )-Vp{B r ) 

Kp(Br) 



(4-4) 



is plotted against p, ^ p ^ 2, for r=k = 0, 1 and 
n = 3, 10. This quantity is independent of cr 2 and 
of the constant c. 

Figure 5 illustrates the possibility that the estimated 
variance of Bk may be substantially biased if k is not 
close to p. For example, suppose that n=\0 and 
the value of p is unknown, but it is known to be in 
the interval (0, 1). If k = 0, to would most likely under- 
estimate V p (Bo), for the relative bias ranges downward 
from 0.0 percent (at p = 0) to about —33 percent 
(at p=l). On the other hand, if & = 1, t\ would be 
likely to overestimate V P {B\), and the bias in this case 
could be as high as 68 percent (if p = 0) of V P (B\). 
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Figure 5. Relative bias of estimator t r , when r = k = 0, 1 . 
R.B. = 100 \E p (t k )-V p (B k )]/V p (B k ) 



5. Effect of the Spacing oi the x Values on 
the Comparison of the Estimators 

Up to this point emphasis has been on the case 
where the x values were n in number and were equally 
spaced on the interval (0, nc); that is, Xi = ic, i — 1, 
2, . . . , n; and to each x there corresponded a single y. 

Now suppose that instead of n different #'s, there 
are N=n/m equally spaced x 9 8 on the interval (0, nc), 
and m y's corresponding to each x; that is, the line 
y = f3x is fitted to the n points, (xi,yij) = (mic,yij), 
i=l,2, . . . ,#,.7 = 1,2, . . . ,m. 

Of course, if m— 1, we are back to the case of n x's 
and a single y for each, so it is a special case of this 
more general form. For the more general case, the 
estimator B p and variances Vp(B p ) and Vp(B r ) have 
the following forms: 



22>;'- p 



B u 



. j=l i= 



N '■ 

2 V ,-2-p 



(5-1) 
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V P (B P ) = - 



(T* 



c 2-p m 3- P y t 2-p 



N 

2 



Vp(B r )=- 



•■2?-* 



i=l 



c 2-p m *-p 



X^ 2 - r 



(5-2) 



(5-3) 



In comparing estimators Bk and B* : +i in section 2, 
the equation, V p (Bk) = V p (Bk + i), was solved for p, and 
the solutions, p n (k, /f+1), are given in table 2. The 
solutions of this equation when the variances are 
of the general form, eq (5-3), are also found in table 2. 
It is the number of x values, N, that determines the 
solution, and so the solutions are found by substituting 
the value of N for the value n of the table. For 
example, if ft =15 and m = 3, then 7V= 15/3 = 5, and 
the solutions are those corresponding to n = 5, i.e., 
0.57 and 1.56. 

In section 3, the relative efficiency of estimators 
was used to show that estimator Bk was better than 
estimator Bk + \ under certain circumstances. The 
same result is obtained for the more general case. 
In addition, any combination of n and m for which 
N is constant gives the same relative efficiency 
(eq (3-1) or (3-2)). Thus, figure 4 is identical 
to those obtained for all pairs n and m such that 
N=nlm= 10. 

The estimator t r (introduced in sec. 4) of the variance 
V r (B r ) can here be written in the form 



SI 



tr = ~ 



- r m 3 - r J; 2 



(5-4) 
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\ 2 2 yi ^~ 



N 



The results given in section 4 on the sign of the bias of 
estimator t r apply in the general case. However, the 
magnitude of the relative bias (R.B.) of t r is affected by 
the choice of N and m for fixed n = mN. 

Table 4 gives the relative bias for n — 10, and shows 
that it is smallest for N= 1 (which holds for all n). 
Table 4 can also be used to calculate the relative bias of 
t r for other values of n. 

From section 4, the R.B. for n observations is given 
by eq (4—4), using eqs (4-3) and (2—1). Let the formula 
obtained from eq (4-4) with Xi = i be denoted by G{n). 
For the general case, the R.B. for N=n/m equally 
spaced x values is found to be 



R.B. (N, n) = 



71-1 



7V— 1 

N 



GOV). 



Table 4 gives R.B. (N, n) for n =10, and also approxi- 
mately R.B. (N, n) for other values of n, where the 



Table 4. Values of V p (B k ), E p (t k ), and R.B. (N, n) for n = 10, a 2 = 
c = 1, and N = n/m 
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E P {t k ) 


R.B. 


V p {B k ) 


E„(t k ) 


R.B. 


V p {B k ) 




10 


0.002597 


0.002597 





0.003306 


0.005550 


68 


0.01550 





5 


.002273 


.002273 





.002778 


.003920 


41 


.00732 




2 


.001600 


.001600 





.001778 


.002025 


14 


.00250 




1 


.001000 


.001000 





.001000 


.00100 





.00100 




10 


.02041 


.01361 


-33 


.01818 


.01818 





.0293 


1 


5 


.01860 


.01309 


-30 


.01667 


.01667 





.0228 




2 


.01440 


.01173 


-19 


.01333 


.01333 





.0150 




1 


.01000 


.01000 





.01000 


.01000 





.0100 




10 


.1709 


.0921 


-46 


.1273 


.0970 


-24 


.100 


2 


5 


.1618 


.0931 


-42 


.1222 


.0975 


-20 


.100 




2 


.1360 


.0960 


-29 


.1111 


.0988 


-11 


.100 




1 


.1000 


.1000 





.1000 


.1000 





.100 



♦When k = 2, Ep{tk)=Vp(B k \ and R.B. = for all p. 

approximation should be fairly good if TV and n are 
large. More precisely, 



R.B.(7V,/m) : 



h{n-\) 
hn—1 



R.B. (TV, n). 



In comparing Bk 9 k = 0, 1, 2, for fixed p, notice in 
table 4 that the differences among the three estimators 
are less pronounced when TV is smaller. Furthermore, 
table 4 illustrates the way in which the choice of spacing 
for ^-values affects the efficiency of the estimators Bk< 
and tk for fixed p and k. 

Thus, it is seen in table 4 that for given p and k, 
V p (Bk) is smallest when N=l. This result is not 
unexpected since it is well known for p = that the 
best estimate of the slope of a line through the origin 
is obtained with all observations made at one point 
as far away from the origin as possible. The same is 
true for any p ^ 2, as follows. When all x\ are equal 
to the largest, x n , then from eq (2-1) 

V p {Bk) = (J 2 lnxr?-P. 

This choice of the x\ minimizes V p (Bk), since 



and 



^Xi 2 -P^n Xn 2 - p i{p^2. 



Minimizing the variance of the estimate of Bk by 
choosing N=l is desirable only when the experi- 
menter does not need observations at intermediate 
points to check on linearity. The dependence of the 
variance of y\ on x\ is irrelevant, of course, when all 
xi are equal; for then all observations y% have equal 
variances. 
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